Acronyms used in this Background and other portions of this document are defined below in the Detailed Description section.
This Background is based on a Wikipedia article titled “Base64”, which is assumed to be largely correct but whose accuracy has not been independently verified by the authors of this patent document. In the Wikipedia article, “Base64” refers to a group of similar binary-to-text encoding schemes that represent binary data in an ASCII string format by translating it into a radix-64 representation. The Base64 term originates from a specific MIME content transfer encoding. Base64 encoding schemes are commonly used to encode binary data to be stored and transferred over media that are designed to deal with textual data. This helps ensure that the data remain intact without modification during transport. Base64 is commonly used in a number of applications including email via MIME, and storing complex data in XML.
The particular choice of character set selected for the 64 characters required for the base varies between implementations. The general rule is to choose a set of 64 characters that is both part of a subset common to most encodings, and also printable. This combination leaves the data unlikely to be modified in transit through information systems, such as email, that were traditionally not 8-bit clean. For example, MIME's Base64 implementation uses A-Z, a-z, and 0-9 for the first 62 values. Other variations, usually derived from Base64, share this property but differ in the symbols chosen for the last two values; an example is UTF-7.
The earliest instances of this type of encoding were created for dialup communication between systems running the same OS—e.g. uuencode, BinHex for the TRS-80 (later adapted for Macintosh® computers, mark of Apple, Inc.)—and could therefore make more assumptions about what characters were safe to use. For instance, uuencode uses uppercase letters, digits, and many punctuation characters, but no lowercase, since it was sometimes used with terminals that did not support distinct letter case.
When the number of bytes to encode is not divisible by 3 (that is, if there are only one or two bytes of input for the last block), then the following action is performed: Add extra bytes with value zero so there are three bytes, and perform the conversion to base64. If there was only one significant input byte, only the first two base64 digits are picked, and if there were two significant input bytes, the first three base64 digits are picked. ‘=’ characters might be added to make the last block contain four base64 characters. As a result, when the last group contains one octet, the four least significant bits of the final 6-bit block are set to zero; and when the last group contains two octets, the two least significant bits of the final 6-bit block are set to zero.
As to padding, the ‘==’ sequence indicates that the last group contained only 1 byte, and ‘=’ indicates that it contained 2 bytes. The number of output bytes per input byte is approximately 4/3 (33% overhead) and converges to that value for a large number of bytes. From a theoretical point of view, the padding character is not needed, since the number of missing bytes can be calculated from the number of Base64 digits. In some implementations, the padding character is mandatory, while for others it is not used. One case where padding characters are often required is when multiple Base64 encoded files are concatenated.
The Wikipedia article states that implementations may have some constraints on the alphabet used for representing some bit patterns. This notably concerns the last two characters used in the index table for index 62 and 63, and the character used for padding (which may be mandatory in some protocols, or removed in others). A table in the Wikipedia article summarizes known variants. Among them, the Char for index 62 may be ‘+’, ‘−’, ‘.’, ‘_’, or ‘!’; the Char for index 63 may be ‘/’, ‘-’, ‘−’, or ‘:’; the pad char may be mandatory ‘=’, none, or optional URL encoding %3D; line length may be variable or fixed; the maximum encoded line length may be 64, 76, application-dependent, filesystem-dependent, language/system-dependent, or XML parser-dependent; line separators may be CR plus LF, or none; characters outside the index table may be either forbidden or accepted but discarded; and line checksums may be absent or a 24-bit CRC which is Radix-64 encoded including one pad character.
According to the Wikipedia article, the first known standardized use of the encoding now called MIME Base64 was in the PEM protocol, proposed by RFC 989 in 1987. PEM defines a “printable encoding” scheme that uses Base64 encoding to transform an arbitrary sequence of octets to a format that can be expressed in short lines of 6-bit characters, as required by transfer protocols such as SMTP. The current version of PEM (specified in RFC 1421) uses a 64-character alphabet consisting of upper- and lower-case Roman letters (A-Z, a-z), the numerals (0-9), and the “+” and “/” symbols. The “=” symbol is also used as a special suffix code. The original specification, RFC 989, additionally used the “*” symbol to delimit encoded but unencrypted data within the output stream.
To convert data to PEM printable encoding, the first byte is placed in the most significant eight bits of a 24-bit buffer, the next in the middle eight, and the third in the least significant eight bits. If there are fewer than three bytes left to encode (or in total), the remaining buffer bits will be zero. The buffer is then used, six bits at a time, most significant first, as indices into the string: “ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/”, and the indicated character is output. The process is repeated on the remaining data until fewer than four octets remain. If three octets remain, they are processed normally. If fewer than three octets (24 bits) are remaining to encode, the input data is right-padded with zero bits to form an integral multiple of six bits. After encoding the non-padded data, if two octets of the 24-bit buffer are padded-zeros, two “=” characters are appended to the output; if one octet of the 24-bit buffer is filled with padded-zeros, one “=” character is appended. This signals the decoder that the zero bits added due to padding should be excluded from the reconstructed data. This also guarantees that the encoded output length is a multiple of 4 bytes. PEM requires that all encoded lines consist of exactly 64 printable characters, with the exception of the last line, which may contain fewer printable characters. Lines are delimited by whitespace characters according to local (platform-specific) conventions.
The MIME specification lists Base64 as one of two binary-to-text encoding schemes (the other being quoted-printable). MIME's Base64 encoding is based on that of the RFC 1421 version of PEM: it uses the same 64-character alphabet and encoding mechanism as PEM, and uses the “=” symbol for output padding in the same way, as described at RFC 1521. MIME does not specify a fixed length for Base64-encoded lines, but it does specify a maximum line length of 76 characters. Additionally it specifies that any extra-alphabetic characters are ignored by a compliant decoder, although most implementations use a CR/LF newline pair to delimit encoded lines. Thus, the actual length of MIME-compliant Base64-encoded binary data is usually about 137% of the original data length, though for very short messages the overhead can be much higher due to the overhead of the headers. Very roughly, the final size of Base64-encoded binary data is equal to 1.37 times the original data size+814 bytes (for headers). In other words, the size of the decoded data can be approximated with this formula: bytes=(string_length(encoded_string)−814)/1.37.
UTF-7, described first in RFC 1642, which was later superseded by RFC 2152, introduced a system called modified Base64. This data encoding scheme is used to encode UTF-16 as ASCII characters for use in 7-bit transports such as SMTP. It is a variant of the Base64 encoding used in MIME. The Modified Base64 alphabet consists of the MIME Base64 alphabet, but does not use the “=” padding character. UTF-7 is intended for use in mail headers (defined in RFC 2047), and the “=” character is reserved in that context as the escape character for “quoted-printable” encoding. Modified Base64 simply omits the padding and ends immediately after the last Base64 digit containing useful bits leaving up to three unused bits in the last Base64 digit.
OpenPGP, described in RFC 4880, describes Radix-64 encoding, also known as “ASCII Armor”. Radix-64 is identical to the Base64 encoding described from MIME, with the addition of an optional 24-bit CRC. The checksum is calculated on the input data before encoding; the checksum is then encoded with the same Base64 algorithm and, using an additional “=” symbol as separator, appended to the encoded output data.
According to the Wikipedia article, RFC 3548 is an informational (non-normative) memo that attempts to unify the RFC 1421 and RFC 2045 specifications of Base64 encodings, alternative-alphabet encodings, and the seldom-used Base32 and Base16 encodings. RFC 3548 forbids implementations from generating messages containing characters outside the encoding alphabet or without padding, unless they are written to a specification that refers to RFC 3548 and specifically requires otherwise; it also declares that decoder implementations reject data that contain characters outside the encoding alphabet, unless they are written to a specification that refers to RFC 3548 and specifically requires otherwise. RFC 4648 obsoletes RFC 3548 and focuses on Base64/32/16, e.g., commonly used Base64, Base32, and Base16 encoding schemes, the use of line-feeds in encoded data, use of padding in encoded data, use of non-alphabet characters in encoded data, use of different encoding alphabets, and canonical encodings.
Another variant called modified Base64 for filenames uses ‘-’ instead of ‘/’, because Unix and Windows filenames cannot contain ‘/’. It could be recommended to use the modified Base64 for URL instead, since then the filenames could be used in URLs also. More generally, Base64 encoding can be helpful when fairly lengthy identifying information is used in an HTTP environment. For example, a database persistence framework for Java objects might use Base64 encoding to encode a relatively large unique id (generally 128-bit UUIDs) into a string for use as an HTTP parameter in HTTP forms or HTTP GET URLs. Also, many applications need to encode binary data in a way that is convenient for inclusion in URLs, including in hidden web form fields, and Base64 is a convenient encoding to render them in a compact way.
According to the Wikipedia article, using standard Base64 in URLs requires encoding of ‘+’, ‘/’ and ‘=’ characters into special percent-encoded hexadecimal sequences (‘+’=‘%2B’, ‘/’=‘%2F’ and ‘=’=‘%3D’), which makes the string unnecessarily longer. For this reason, modified Base64 for URL variants exist, where the ‘+’ and ‘/’ characters of standard Base64 are respectively replaced by ‘−’ and ‘_’, so that using URL encoders/decoders are no longer necessary and have no impact on the length of the encoded value, leaving the same encoded form intact for use in relational databases, web forms, and object identifiers in general. Some variants allow or require omitting the padding ‘=’ signs to avoid them being confused with field separators, or require that any such padding be percent-encoded. There are other variants that use ‘_−’ or ‘._’ when the Base64 variant string is to be used within valid identifiers for programs.
XML identifiers and name tokens are encoded using two variants: ‘.−’ for use in XML name tokens (Nmtoken), or ‘_:’ for use in more restricted XML identifiers (Name).
Another variant called modified Base64 for regexps uses ‘!−’ instead of ‘*−’ to replace the standard Base64 ‘+/’, because both ‘+’ and ‘*’ may be reserved for regular expressions (note that ‘[ ]’ used in the IRCu variant above would not work in that context).
According to the Wikipedia article, atob( ) and btoa( ) methods defined in a HTML5 draft specification provide Base64 encoding and decoding functionality to web pages. The atob( ) method is unusual in that it does not ignore whitespace or new lines, throwing an INVALID_CHARACTER_ERR instead. The btoa( ) method outputs padding characters, but these are optional in the input of the atob( ) method.
According to the Wikipedia article, Base64 can be used in a variety of contexts. For example, Base64 can be used to transmit and store text that might otherwise cause delimiter collision. Spammers use Base64 to evade basic anti-spamming tools, which often do not decode Base64 and therefore cannot detect keywords in encoded messages. Base64 is used for PHP obfuscation. Base64 is used to encode character strings in LDIF files. Base64 is often used to embed binary data in an XML file, using a syntax similar to, e.g. favicons in Firefox's bookmarks.html. Base64 is used to encode binary files such as images within scripts, to avoid depending on external files. The data URI scheme can use Base64 to represent file contents. For instance, background images and fonts can be specified in a CSS stylesheet file as data: URIs, instead of being supplied in separate files.